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This study contributes to provide an enhancement technique for improving 
the Iterative Back Projection (IBP) Super Resolution technique by using the 
Sharp Infinite Symmetrical Filter (SISEF). Theoretically, the IBP technique 
operates as minimizer the error reconstruction iteratively until it refined the 
High Resolution (HR) image. However, due to iterative manner and lack of 
edge guidance during the back projection operation, this technique has 
suffered from produced the ringing artefact on the HR image appearances. 
Additionally, the IBP reconstruction also demands for large number iteration 
for accomplishing the prediction HR image. This problem arose when the 
IBP estimator tended to oscillate at the same solution frequently. In order to 
overcome these constraints, the SISEF is deployed as regulator to improve 
the IBP estimator with provides an accurate edge information for enhancing 
the edge image and reduce the ringing artefacts. Fortunately, highly precision 
of edge information provided by SISEF capable to reduce amount of 


estimation process repetition. 
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1. INTRODUCTION TO SUPER RESOLUTION ENHANCEMENT 

The area of this study is getting a lot of interests from researcher since the conventional Super 
Resolution (SR) technique suffers from imperfection in the High Resolution (HR) image estimation. 
In Chiang et al [1] works, an improvement of IBP is produced in the resampling algorithm by integrating the 
image warping algorithm inside the IBP technique. Moreover, they proposed the implementation of coupling 
the degradation model of imaging system into integrating resample to produce better estimation. In Cohen et 
al [2] work, an improvement is suggested through applying Polyphase filter to replace the geometric trans- 
formation inside the Iterative Back Projection (IBP) and improved the image resolution. Furthermore, 
the edge enhancement technique proposed by Hyunwoo et al [3] using an Anisotropic Diffusion to suppress 
the noise and highlighted the edges of image. 

On top of that, the image enhancement is done in Yan et al [4] works. The combination of Papoulis- 
Gerchberg extrapolation inside the IBP reconstruction is proposed to enhance the spiral Computed 
Tomography (CT) slice image. Besides that, Qin suggested the efficient improvement in the IBP algorithm 
through improvement of the first initial guess which is estimated by wavelet locally adaptive algorithm 
compared to normal interpolation method [5]. Next Liang et al [6] also gave improvement in the edge 
detection during conducting the initial interpolated image via apply Non-Local IBP fast algorithm. 
In the Bengtsson et al [7] proposed L*a*b domain for improving the Human Visual System in the HR image. 
This technique is capable to improve the output of HR image by preserving the edge information. 
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In order to recover the HR image, Bareja et al [8] proposed a combination of IBP algorithm with 
Canny Edge Detector and error difference image to recover loss of frequency information. The improvement 
is continued in Nayak et al [9], by proposing a hybrid technique between IBP technique and Cuckoo 
optimizer to enhance the initial guess and HR output image. In Pejman et al work [10], the improvement in 
the IBP algorithm is proposed through applying the interpolation of LR images before registered inside the 
IBP algorithm. Another work in Cheref et al [11] introduced edge enhancement by using the Anisotropic 
Diffusion technique to preserve any possible edge image. In Mei et al [12] works, implementation of adaptive 
Perona Malik as edge preservation is done in SR reconstruction. 

The enhancement works in the IBP is continued with reducing the ringing artefacts by applying a 
variant and local variance variant based on adaptive analysis [13]. Moreover, the quality of visual output 
reconstruction image is improved through applying the sparse linear regression and IBP. Additionally, 
modification in Gaussian high pass filter is done to refine the initial guess [14]. Furthermore, the accuracy of 
reconstruction can be increased when the residual error is enhanced by implementing the Anisotropic 
Diffusion with the help of IBP estimator to produce fine HR image with edge preservation technique [15]. 
In Makwana et al [16] proposed a combination of IBP algorithm with Canny edge detector and Gabor filter 
for improving the error minimization during back projection operation. Besides that, Nayak et al [17] 
provided the edge image improvement in the IBP technique by using the P-Spline and MuCSO-QPSO 
algorithm. The P-Spline provides sharp edge information and MuCSO-QPSO optimized the 
estimation method. 

The Iterative Back Projection technique has ability to minimize the reconstruction error efficiently 
in iterative manner. This error reconstruction is back projecting to HR grid to correcting the next estimation 
image. Although this technique improves the image intensity with removing the blur effect. However, 
the iterative routine tends to produce the ringing artefacts along the edges because of back projecting is 
without any edge guidance. In the present work, this study proposes the edge enhancement technique inside 
the Iterative Back Projection technique. This edge enhancement technique is combined between the IBP 
reconstruction and Sharp Infinite Symmetrical Exponential Filter (SISEF). The SISEF function as sharp 
edges detection and then preserves the possible edges detected by re-highlights the edges inside the 
estimation process. Coincide, the information supplies works as edge guidance during the back projection to 
avoid across edge projection thus the ringing effects can be removed. This sharp edge detected capable to 
enhance the reconstruction HR estimation with reduce the error reconstruction and preserving the lost 
frequency information with robust toward ringing effects and rapid estimation process. 


2. RESEARCH METHOD 
2.1. Iterative Back Projection 

The main idea of the Iterative Back Projection (IBP) technique is to reconstruct the High Resolution 
(HR) image from observing the multiple degraded input images. The estimation done by projecting back the 
difference between simulated Low Resolution (LR) images and the observed LR images [18]. The estimation 
keeps continued until the HR image generated which has same LR images with observed LR images after 
passing the same blurring and down sampling process. Basically the IBP method can be divided into two 
phases. In the first phase, it generates the simulated image and then projected back the error from the 
difference between simulated and observed LR images to reconstruct the HR estimation [19]. 
The construction of simulated the LR images Y;, can be formulated in 1. 


Y, = D, By Wf (1) 


This simulated LR image ¥;, has number k*" of image and the pixel intensity of LR images derived 
from the HR image f after get influenced by the degradation function. The degradation function is a 
combination of blur kernel function B, and translation wrapping function W,, influenced before getting the 
down sampling by the decimation operator D. 

In the reconstruction of IBP process, this technique starts with computing the initial guess f© HR 
image via applies the average method. Then, the imaging process determines the set of simulated LR image 
Y,,©. However, the initial guessing process may provide many solutions than one accurate solution. 
The selection of accurate the initial guess method could driving the algorithm to determine an unique, faster 
and smooth solution. A good selection of an initial guess method was suggested by Irani is the average 
method of the attributes of information from registered LR images observed [18] such in the 2. 


fO = Th We) 2) 
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If the initial guess HR image f is matching enough with the reference image, then the simulated 
images Y,° = D,B,W,f should be identical with the observed images Y, and no more correction is 
required. If the difference images arise between Y, and Y,°, then the improvement the initial guess by back 
projecting the error with combining the current information in the HR grid. This step is yielding an 
improvement to the next HR image f such expressed in the 3. 


ft = 4-000, [h - Of] (3) 
Where 
Qi = Dy BW, (4) 


This procedure keeps repeating in iterative manner until it meets the local minimum error. 
The error function can be described as in 5. 


et = - WoallYe — Yell; (5) 


2.2. The Infinite Symmetrical Exponential Filter (ISEF) 

The Infinite Symmetrical Exponential Filter or Shen Castan Edge Detector is an optimal smoothing 
filter with a very simple recursive algorithm. The core concept of this filter is implementing the 
differentiation of optimal exponential. It gives a good impact during edge detection when it has less responds 
to noise and give precision in edge localization. Generally, the problem facing of Laplacian Gaussian filter 
during the edge detection which it has contradiction between the noise insensibility and the precision 
localization. Thus to overcome this problem, the ISEF proposed the linear filter based on one step model and 
multi edge detection. 

This optimal smoothing filter is a symmetric exponential in infinite large window size and very 
simple recursive algorithm. The difference between input and output of this recursive filter has the capability 
to detect the edges with less sensitivity to noise and high precision in localization. The ISEF filter has 
proposed two methods for edge detection either can be used the maxima of the gradient (GEF) or the zero 
crossing of second directional Derivative along the Gradient (SDEF). The ISEF filter produced the GEF and 
SDEF from the first and second derivatives of the operator for symmetric exponential filters respectively. 
Initially, the normalized symmetric exponential filter in one dimensional (1D) is written in 6. 


fix) = Cf) + fox) — a9 d(x) (6) 
Where 


1 


fs (7) 


~ (2=a) 





The exponential is a symmetrical function which is can be divided into the right side function f, (x) 
and the left side function f,(x). Thus the second derivative of exponential function for the SDEF along the x 
axis can describe in 8. 


Ixy) = 25 (I@ey) * Fy) (8) 
Where in the discrete form of SDEF is written in 9. 
lex) = 10,Y) * f 0) * f20) * (40) + 0) — 20) * F,0) * 20) (9) 


The equation above also can be used to calculate the gradient along the y axis direction. 
For instance, this equation scheme can be illustrated in the process flow Figure 1. 


2.3. Previous Works in Edge Enhancement Technique 


The edge enhancement technique provides additional correction information during the 
reconstruction computation based on the guided edge information detected. The information of edge 
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detected, then is used to strengthen back the lost edge information. Additionally, this process provides an 
edge guided to the IBP technique during the back projecting to HR grid size to avoid a ringing effect 
disturbance. Furthermore, the efficiency of reconstruction process is influenced by the precision the edge 
detection provided. This high precision of localization and high rate detection are important features to 
increase the rate of edge detection and at the same time, it helped to increase the rate of correction process. 
This study meets two good technique have implemented with IBP in edge enhancement technique such 
Anisotropic Diffusion and ISEF filter. 


I(x, y) 











1.) Tee GY) 


Figure 1. ISEF process flow 


2.4. Propose IBP technique with Sharp ISEF regulator 

This study has concerned on the edge detection is provided by the ISEF method. This Shen-Castan 
method has some advantages such as yields edge detail information with less responds to noise and high 
precision edge localization. This study also interests to use edge detection through zero crossing provided by 
the Second Directional Derivative Fillter (SDEF). This filter is based on exponential function which gives 
better performance than the Gaussian based filter. Hence, this technique could provide much better edge 
preserving [20]. The high accuracy in SDEF detection supplies much possible additional information 
especially about high frequency information. This precision on information helps the estimator to make 
predictions and preserves the weak and loss of edges more efficient. 

As aforementioned earlier, the IBP method performed well in minimizing the residual error during 
the HR estimation. The estimation is done by back projecting the difference of observed image and simulated 
image to the HR correction process. This iterative process keeps on repeating until no more error cost 
function arises without any edge guidance. Consequently, estimated the HR by the IBP algorithm is not able 
to fully recover the lost edge information. Thus, the ISEF method is the best solution to improve the de- 
blurring process and sharpen the edge details. This ISEF implementation can be illustrated as in the Figure 2. 

According to the proposed algorithm in Figure 3, the initial guess gets the first enhancement from 
the implemented ISEF. This process is important since the observed LR will be magnified to the HR grid, 
thus some information will be lost and blurry image is generated. Also, this stage gives improvement in the 
quality of initial guess image to get close to the possible to optimum estimation. As the HR initial guess has 
good quality, the IBP reconstruction only requires few efforts for enhancing process. Besides increasing the 
quality of HR estimation, this stage also contributes to save a lot if resources in a faster way. The first refine 
process in the initial guess can be expressed in 10. 


fo =f +A? * Hise) a) 


Where A is constant for the stability equation. f° is the first initial guess projected from the observed LR 
images by using the average method. In order to enhance the f°, the SDEF method in the ISEF observes the 
priori information initial guess to produce the gradient information. This gradient contains an edge 
information which is then fused inside the current estimation image. This additional edge information 
provides recovering of the weaker frequency information and strengthens back the structure image. 
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Basically, the residual information is generated from the difference of simulated and observed LR 
images which are used to correct the current HR estimation. Meanwhile, the correction process can be 
optimized when adding the extra edge information. In Patel [21] work, the additional gradient information is 
directly fused inside the residual information. However, this approach has some differences since additional 
information needs a step size or weighting factor to ensure this combination converges to the optimal local 
minima faster. The suitable value of step size is crucial criteria to ensure the estimation achieves optimum 
HR output. Besides that, it also gives improvement in the number of repetitions on of the system. 
This calculation for step size can be referred in 11. 


2 
—— 1 
OSES aC Ch) 


Where Hf (x) is the Hessian function of matrix of f at x and ||-|| is the spectral operator norm which used to 
find the largest eigenvalue of function f/ The parameter t is known as the step size for gradient of image 
edges computed. 

In order to determine the optimal result, this optimal residual information is then projected back to 
HR grid. This scenario can be described in mathematical as such 12. 


fear = fe tr’ + T1(f — fe) * ISEF — (fe-1) * ISEF] (12) 


In Patel works, the enhancement of the edges is taken from the difference of computed initial guess 
and current HR. This difference provides a guidance to the edge intensity and the precision in the edge 
localization. However, the precision edge localization can be increased by shrinking the width edges to 
become sharper. Thus, the accuracy in edge localization contributes to more accurate estimation, as well as 
an increment in the quality of output and decrease the number of iterations for estimation process. 
The sharper edge process can be done by subtracting the difference edge (f — f,) with edge detected from 
the previous state HR image(f,_,). Both of detected edges are determined by the SDEF method just before 
the subtraction operation. This situation can be illustrated such in Figure 2. 


combination 





(a) (b) (c) 


Figure 2. Sharper gradient image (a) difference edge of HR (b) edge of priori HR (c) proposed sharp edge 


3. RESULTS AND ANALYSIS 

This section presents the experiments and results obtained by the proposed combination between 
IBP with SISEF edge enhancement technique. These experiments use the MATLAB platform as experiment 
simulator. These experiments also use the standard image form ISI and DESCAI database with the standard 
image size is 256x256 resolution. Firstly, this experiment creates a sequence four LR images. Each of LR 
images has shifted from original image by a pixel in the vertical direction. Then, these images applied effect 
of camera PSF. The effects involved in this process contain of blurry and decimation operation. Then this 
shifted LR image convoluted with the Gaussian low pass filter of size 3x3 with a standard deviation of 3. 
Then resulting image was down sampling by factor 2 in each direction. Therefore, the same operation with 
the different motion vectors in vertical and horizontal directions is used to produce four LR images. 
The proposed technique measurement based on the highest PSNR recorded. Besides that, the cost 
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computation complexity measurement is used based on number of iterations consuming for finishing the HR 
estimation. The results and analysis are equipped the comparison with the Cheref work [11] and Patel work 
({21] along the work. 


3.1. Situation: Blurry Effect Information Perfectly Known 

The result in Table 1 shows that the combination IBP with proposed technique Sharp ISEF regulator 
(SISEF) is leading for most of PSNR results. By looking at the average result, this technique gives an 
improvement about 1.75 dB compares than ordinary IBP technique and this result is a higher quality 
improvement is provided compare to other techniques. This means, the SISEF proposed technique has the 
capability to provide a better edge enhancement. This edge detector equipped with higher localization 
precision and less error detection, therefore in most cases of image tested this proposed ISEF has produced a 
better performance. 

Besides that, by comparing with the Patel technique. This proposed technique successfully to 
increase the PSNR up to 0.758 dB than Patel technique. Furthermore, by referring in Figure 3, this SISEF has 
successfully increased the precision and accuracy of edge priori information from the conventional ISEF 
method in Patel work. Thus, this contribution helps the IBP estimator predicts the HR construction more 
precisely and accurately. Additionally, Figure 4 has shown the SISEF has successfully reduced the ringing 
artefacts around the edges of the HR image compared with the ordinary IBP technique. Perhaps, this situation 
helps to increase the appearance of HR image. 

Despite of the good performance, this technique has difficulty during handling the critical image 
such the image contains a lot of high frequency information. For example, the Mandrill image contains a lot 
of high frequency information such hair around the mandrill face. This disadvantage rose when the SISEF 
detector only traces the edges in horizontal and vertical direction only. Hence, in some direction this detector 
failed to provide details high frequency information. 
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Figure 3. The IBP with lorentzian ISEF regulator process flow 
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Figure 4. Ringing artefacts (a) IBP output contaminated with the ringing artefacts (b) proposed technique has 
reduced the ringing artefacts 


Table 1. PSNR for Multi Kinds of Input Image 





The IBP technique with 

IBP Patel Cheref CANNY SISEF 
LENA 27.730 28.651 28.034 28.018 28.952 
CAMERAMAN _ 26.477 27.220 27.319 26.758 27.224 
MANDRIL 23.284 23.464 23.486 23.336 23.273 
PEPPER 25.941 27.956 29.023 26.391 30.253 
CLOWN 24.560 26.673 27.932 25.067 28.784 
JETPLANE 26.020 27.177 27.571 26.353 27.648 
BRAIN 28.041 30.131 31.319 28.699 32.533 
SHIP 28.923 29.494 29.714 29.188 29.435 
USAF 22.670 22.183 22.624 22.656 22.183 
BRIDGE 23.982 24.648 24.824 24.178 24.890 





On the other hand, in some cases the technique proposed by Cheref also provides the highest quality. 
This technique performed well during handling difficult image such Mandrill and USAF image. 
The Anisotropic Diffusion in Cheref works has the capability to seek the edge in many directions (north, 
south, west, and east). Thus, this method has the capability to provide the detail edge information and give a 
promising result in the difficult image. Moreover, this edge detector has good rate detection and minimum 
responds toward reduction edges. Thus, this technique is among the best edge enhancement technique. 

The Gaussian based filter design such as Canny techniques only provides a little improvement. 
This condition happens when the Gaussian based filter has a contradiction function between providing 
smooth function and precise localization. Furthermore, the Canny edge detector can provide a very sharp 
edge image, but the intensity edge detected is small. Normally, this type of filter has less precise localization 
than ISEF filter. 


3.2. Number of Iterations 

In terms of the computational complexity, the IBP with edge enhancement technique measurement 
is based on the number of iteration algorithm for accomplishing the estimation. The main ideas for this 
improvement depends on the accuracy of edge information provided. This condition helps the optimizer tools 
make a decision during cost computation. The accurate additional information avoids the estimation tool for 
predicting the same solution frequently. As a result, this edge enhancement transforms the IBP technique 
more efficient in predicting and light computational. 

Gratefully, the proposed technique capable to boost up the Gradient Descent performance better than 
previous. In additional this technique can speed up the cost computational until 70% faster from the 
conventional Gradient Descent method. This situation proved that the edge enhancement technique assists to 
optimize the estimation process during seeking the final HR image. The accurate estimation function depends 
on the precision of edge detector provides the high localization and high rate edge detection. This accuracy 
edge detection helps the estimator to predict further step to getting close the optimal result. Table 2 shows the 
number of Iterations 
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Table 2. Number of Iterations 








The IBP technique with 
IBP Patel Cheref_ | CANNY _ SISEF 
LENA 89 46 45 72 30 
CAMERAMAN 82 46 44 67 27 
MANDRIL 117 65 58 99 35 
PEPPER 116 62 57 96 31 
CLOWN 114 63 57 94 39 
JETPLANE 92 47 48 76 35 
BRAIN 91 51 44 73 22 
SHIP 78 41 41 63 25 
USAF 44 27 30 37 25 
BRIDGE 98 56 50 81 34 





Another key point for speed up the estimation depends on the accurate rate of step size for the 
gradient. In reality, the gradient value detected varies large intensity amount, thus the estimation never be 
converging after receiving the large amount of data. This situation can be handled by sets the suitable rate for 
step size. The selection of step size is important to ensure estimator receives an optimum information with 
less computational cost. In this experiment all technique examines provided same optimum step size expect 
the technique proposed by Patel. Thus, this estimation efficiency totally depends on the performance of edge 
detector performance. The result in Table 4.4 shown, the Sharp ISEF edge detector is the best regulator 
during performing the estimation process since this technique provides lesser iteration compare the other 
regulator technique. Additionally, the proposed technique and the Patel technique use same edge detector. 
However, the suitable step size rate gives an advantage of proposed techniques to become light cost 
complexity up to 40% faster. 


4. CONCLUSION 

This study proposed a precise edge enhancement regulator to improve the IBP reconstruction 
technique problem by using the Sharp ISEF regularization. Normally, the IBP reconstruction technique has 
suffered from produced the ringing artefacts on HR estimation and it also required a large number of 
iterations to accomplish the estimation. Moreover, this technique leak of capability to preserve the lost high 
frequency information due to blurry effect. However, the Sharp ISEF has a solution to cure the IBP 
reconstruction problem with providing precise edge enhancement with light cost computation. The Sharp 
ISEF method has highly precision localization and less error rate detection. Thus these advantages help the 
estimator to compute the correction process more accurate and faster. 
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